
You might be familiar with the Windows Task Manager - the go-to tool for monitoring and managing processes. In the Linux world, the equivalent command-line tool is top. While there might not be a fancy GUI, the power of top lies in its simplicity and real-time updates.

In this post we'll dive into `top` command (if you can understand `top` you can easily analyse `htop` )

Most Linux distributions come preinstalled with the `top` command, offering an instant peek into your system's performance. You can install the `htop` utility separately if you like pretty colours in your life.

now run

```bash
top
```

![top](https://i.imgur.com/1DCd3Qo.png)

### Decoding the Dashboard

Now, let's take a closer look at the top-left corner of the `top` interface:

- **top:** This header indicates the overall system up-time, the number of logged-in users, and the system load average. It's your dashboard's welcome mat, offering key insights at a glance.

- **Tasks:** This section is pretty self explainatory, provides an overview of the different states of processes on your system: running, sleeping, stopped, and even the "zombie" processes. (Don't worry, if you don't know what is a zombie process head [here](https://www.geekbits.io/understanding-zombie-processes-in-linux/#:~:text=A%20zombie%20process%20refers%20to,there%20as%20a%20zombie%20process.) and learn more about it)

- **%Cpu(s):** Ah, the heart of the matter. Your CPU's performance breakdown is split into eight key parts:

  1.  us (User): The percentage of CPU time spent executing user-level processes. This consists most of the applications you interact with.
  2.  sy (System): This chunk represents the CPU time dedicated to kernel processes - tasks critical to the system's operation.
  3.  ni: The "nice" value. It reflects the percentage of CPU time given to processes with adjusted priority levels. (You can ignore it like how u ignore your academics)
  4.  wa (Waiting Time): When the CPU waits for input/output operations to complete, this percentage is increased.
  5.  hi: CPU time spent handling hardware interrupts. A crucial role in managing hardware communication (like keyboard, Mouse etc...)
  6.  si: Similar to "hi," but this time handling software interrupts.
      If you don't know what are interrupts, head [here](https://www.techtarget.com/whatis/definition/interrupt#:~:text=An%20interrupt%20is%20a%20signal,service%20or%20a%20current%20process.) to learn more about it
  7.  st: If your machine has shared access or you run a VM inside the os this indicates the time your CPU is being "stolen" by a virtualized guest operating system.

- **Mib Mem:** This segment gives you a vivid picture of your system's memory like insights into how memory is allocated and utilized.
  - Total: The total physical memory available to your system. This includes RAM and other memory resources.
  - Used: The amount of memory currently in use by various processes and the operating system.
  - Free: The memory that's currently unallocated and available for use.
  - Shared: Memory shared among multiple processes, providing an efficient way to conserve resources.
  - Buffers: Memory used for storing data in transit between devices or applications.
  - Cached Memory used for caching data from disk storage, enhancing system performance.
- **Mib Swap**: This metric refers to the virtual memory - a extension of your physical RAM.
  Again if you don't know what is swap memory feel free to read more about it [here](This metric delves into the realm of virtual memory - a dynamic extension of your physical RAM.).
- **Total:** The total swap space available, which acts as an extension of your physical memory.
- **Used:** The portion of swap space currently in use by processes and the system.
- **Free:** The unallocated swap space awaiting future usage.

Now Coming to the table, each row offers a snapshot to processes stats

![top-2](https://i.imgur.com/Bg4zMVj.png)

1. **PID (Process ID):** The unique identification number assigned to each process.
2. **USER:** The username of the user who initiated the process.
3. **PR (Priority):** The priority of the process, reflecting its importance in the system's task queue.
4. **NI (Nice Value):** A user-assigned priority adjustment, determining how much CPU time the process receives.
5. **VIRT (Virtual Memory):** The amount of virtual memory used by the process, including shared libraries and code.
6. **RES (Resident Memory):** The non-swapped physical memory used by the process.
7. **SHR (Shared Memory):** Memory shared with other processes, reducing duplication.
8. **S (Status):** The current process status: R (Running), S (Sleeping), Z (Zombie), and more.
9. **%CPU:** The percentage of CPU time consumed by the process since the last update.
10. **%MEM:** The percentage of physical RAM used by the process.
11. **TIME+:** The total CPU time consumed by the process, including both user and system time.
12. **COMMAND:** The name and arguments of the command that spawned the process.

You might ask what is the use of learning all this, when your system lags or is slow you can use this to determine what is the exact problem

How these parameters can be used??

- High I/O Wait Time: If any process is waiting for high I/O bound time then it might indicate that the memory is not able to access faster (eg: postgres on small systems with volumes mounted)
- CPU Intensive Processes: If any process is CPU intensitive then the us number will be very high then proabably need to optimise the program to adapt to the hardware specs or upgrade the hardware.

* Resource Hogs: Keep an eye on processes with high `%CPU` values. These may be consuming excessive CPU power.
* Memory Eaters: Processes with substantial `%MEM` values could be using up your physical memory.
* Virtual vs. Resident Memory: Compare `VIRT` and `RES` to gauge a process's memory usage efficiency.

There are so many insights that can be gained from just analysing the `top` window that can be used to identify bottlenecks, and make informed decisions about optimizing your system's hardware, software, and configuration.

Now go and try it, after `top` install `htop` and make your life easier :)


Diving Deep into top & htop


Vagrant is a tool to build up and operate Virtual Machines on the top of innumerable (actually few) virtual platform over an operating system, what do I mean by this?

We all have used oracle virtual box for quickly spinning up a VM and prototyping on it, then deleting the entire instance once we are done.

Let's be honest it is a tedious process which involves you setting up the hypervisor (in case of windows), virtual box (Linux) downloading the ISO files configuring the network and the boot up time.
`
We get oracle virtual box or VM ware workstation (community edition) which includes a Graphical User Interface which in turn adds extra overload to process and memory. Which is never used most of the time. I go crazy each time I have to set up a new OS.

What If I told you there is a saner way of doing all these while you don't go crazy, I Introduce you to Vagrant, a HashiCorp developed tool to solve the exact problems I mentioned.

## Vagrant

> Vagrant enables the creation and configuration of lightweight, reproducible, and portable development environments. - HashiCorp

You can think vagrant similar to docker, it requires a specification file where all the necessary specification is configured, then it will automatically download the required OS image and spin it up and did I say it also provides you **SSH** access to the VM out of the box.

Vagrant seamlessly collaborates with multiple virtualization tools on various operating systems. You'll find options like Hypervisor, VirtualBox, and my personal preference, libvirt. Now, if you're unfamiliar with the term "virtualization tool," you can explore more about it [here](https://www.redhat.com/en/topics/virtualization/what-is-a-hypervisor) and [here](https://www.makeuseof.com/best-virtual-machine-linux-kvm-virtualbox-qemu-hyper-v/).

I personally like `libvirt`due to its efficient resource utilization and minimal overhead. However, there's a downside—there aren't as many official OS images supporting libvirt as a virtualization tool. But fear not, the community has contributed viable customizations and images, making it a robust choice. Anyone can customize the OS to work on the top of it and upload it to the community images of the vagrant.

Enough talking, let's get into it :)

First you need to install vagrant and choice of your virtualization tool, (if you are a beginner I advise you to go ahead with VirtualBox)

## Prerequisites

- You can find the vagrant installation details [here](https://developer.hashicorp.com/vagrant/downloads).

- Install a virtualization product such as; [VirtualBox](https://www.virtualbox.org/), [VMware Fusion](https://customerconnect.vmware.com/downloads/get-download?downloadGroup=FUS-PUBTP-2021H1), or [Hyper-V](https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v).

Vagrant requires a configuration file named `VagrantFile` (without any extension) where we'll decide all the configurations for the VM.

```
vagrant init
```

The above command automatically initializes the initial `VagrantFile` which has all the parameter's commented out

_(refresh the page if you don't see the code snippet below)_

<script src="https://gist.github.com/Prajwalprakash3722/fa046fe832f947cfa5081eef320119c0.js"></script>

Let's try to understand what is going on

```
Vagrant.configure("2") do |config|
```

This is the version of the Vagrant that should be used

```
config.vm.box = "base"
```

This is the specification of the base OS that VM, Think box as an Operating System which is already configured

since I am using `libvirt` as my tool, I need to select the boxes that support `libvirt`, you can do that [here](https://app.vagrantup.com/boxes/search?provider=libvirt), as I told there are not many offical boxes that support `libvirt` as provider, but the community ones are fairly good and are stable.

Port forwarding can be also done

```
config.vm.network "forwarded_port", guest: 80, host: 8080
```

Here whatever service that is exposed in the machine on port number 80 is forwarded to your machines `8080` port.

```
config.vm.network "private_network", ip: "192.168.33.10"
```

This configuration allows you to set up a public network interface for your VM. This means that your VM will have its own IP address on the same network as your host machine. This is useful when you need to expose services running on the VM to the outside world.

```
config.vm.synced_folder "../data", "/vagrant_data"
```

Synced folders allow you to share files between your host machine and the VM. This is incredibly useful for development, as you can edit code on your host machine and have it immediately available within the VM. The first parameter specifies the path on your host machine, and the second parameter specifies the path within the VM, Very Similar to docker volumes or Kubernetes Persistent Volume.

```
config.vm.provider "libvirt" do |vb|
  vb.gui = true  # Display the VirtualBox GUI when booting the machine
  vb.memory = "1024"  # Customize the amount of memory on the VM
end
```

This section allows you to configure provider-specific settings for libvirt. You can customize the amount of memory allocated to the VM.

Now with all the above knowledge lets bring up 2 ubuntu VMs inside our system.

<script src="https://gist.github.com/Prajwalprakash3722/d200bf2997dc55e856ca116f53ef1f55.js"></script>

To bring the VMS up, we need to tell vagrant to find the file and bring up the containers

```bash
vagarnt up
```

Now Vagrant will download the ubuntu 22.04 generic box from the hashicorp cloud and start to provision the defined resources.

After all you should be seeing a ` Machine booted and ready!` message then you can use the machines, but how to access them?

Vagrant provides a special way of sshing to the VM's created using public and private key that is automatically configured while bring up the Vm's.

Now we have 2 Vm's namely - ubuntu1 - ubuntu2

Let's ssh to the first node,

```
vagrant ssh ubuntu1
```

this will login you to the Vm and you can just use the VM as a normal machine make sure to port-forward if you want to access the services running on Vms from the host system.

When done, either permanently destroy them:

```
vagrant destroy
```

(Note: This will permanantely destroy your Vm and related data unless you have a synced folder)

Or suspend them for later:

```bash
vagrant suspend
```

To resume a suspended VM:

```bash
vagrant reload
```

That's Vagrant in a nutshell, hope you liked it.

cheers :)


Quickly spin up VM's using Vagrant


## Diving deep into my Personal Infrastructure

We are 110 days in to 2023. Have you been able to keep your New Years Resolutions or goals, that you so keenly set back on Jan 1? Yea, its hard eh? We all have the best of intentions but once reality sets in, its hard to get out of our already-established routines.
I shared some of my goals for 2023 [here](https://blog.devcoffee.me/posts/look-ma-its-already-2023-and-i-am-dumb).

One of them was to setup a reliable personal infrastructure, I have been working on this for a while now and I am happy to say that I am almost there. I have been able to setup a reliable infrastructure that is completely customised for me.
I didn't want to compromise on security, privacy, or accessibility.

Before I dive into the details, Let me give you a quick overview of all the services I use in daily life.

- Github
- Github CI/CD
- Vercel (for hosting)
- Google Password Manager
- Google Drive (for backups)
- Calorie Counter App
- Some Platforms for Deploying Backend Services and Databases & so on..

So now, let me tell WHY the hell do I need a personal infrastructure?, I am just a student, not some big startup

I’m a person who’s only satisfied if I feel I’m being productive. I like figuring things out.
I like building things. And I want to do as much of that as I can. And part of being able to do that is to have the best personal infrastructure I can have.

This helps me to figure out how to do things, how to build things, and how to make things better. And that’s what I want to do.

Let's dive into the details.

I have 5 main goals for my personal infrastructure.

1. **Security** - I want to be able to keep my data safe and secure. I don't want to compromise on security.
2. **Privacy** - I want to be able to keep my data private. I don't want to compromise on privacy.
3. **Accessibility** - I want to be able to access my data from anywhere. I don't want to compromise on accessibility.
4. **Reliability** - I want to be able to rely on my infrastructure. I don't want to compromise on reliability.
5. **Customisation** - I want to be able to customise my infrastructure. I don't want to compromise on customisation.

I have 3 VPS hosted on oracle cloud, 1 Raspberry Pi 3b+ and 1 old CPU with `2x Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz` and `8GB RAM` and `1TB` storage.

So all the VPS are running on `Ubuntu 20.04 LTS` and the Raspberry Pi is running on `Raspbian`. and the cpu is running on `Fedora Server Edition 37`.

Now how to interconnect all these devices? You guessed it right, I use `VPN` to interconnect all these devices. In my case, I use tailscale which according to me is the best out there in market.

All the 3 VPS have a public IP address, but not the Raspberry Pi and the CPU. I use `Nginx` as a reverse proxy to route all the traffic to the correct VPS. Basically it pipes the traffic from public IP to the private IP, one of the VPS acts as a gateway to the internet.

[![network-diagram](https://mermaid.ink/img/pako:eNqFUstuwjAQ_BVrT60UEI4hITn00l6QWmSB2kOViyFbsErsyHEoKfDvdR7i0UrFkWzvzmR2PfYeljpFiGFlRL4mz7NEETcmyqJRaEmv93Dg5WIjl2TCD-SNz-ldAm4hNIH7llwnG2KqMyEVUSLDA5mupNo5brOSGW7RFEi40bvq9GeLNTWM3AqLTZFH_vofPBNFvkBjKi6d_CkiXN7QdX36XfP-bSrrqKyh1l9NL8pF61RnyhTtlzafLXjp3DlT29NGqNJ281urq_xH7OREPS7OfSXuX0XsohR4kKFxd5K6C97XQAJ2jRkmELvtRq7WNoFEHR1RlFbPK7WE2JoSPSjz1LX0JIVrMYP4Q2wKl82Fetf6KoZ4DzuI6WDUH4eMhW6ijPoB9aCC2A9G_SAMI0qZA6KABUcPvhsJ2h-xAQ2jQeQPx8EwGPseYCqtNi_tk2xe5vEHZE3JvA?type=png)](https://mermaid.live/edit#pako:eNqFUstuwjAQ_BVrT60UEI4hITn00l6QWmSB2kOViyFbsErsyHEoKfDvdR7i0UrFkWzvzmR2PfYeljpFiGFlRL4mz7NEETcmyqJRaEmv93Dg5WIjl2TCD-SNz-ldAm4hNIH7llwnG2KqMyEVUSLDA5mupNo5brOSGW7RFEi40bvq9GeLNTWM3AqLTZFH_vofPBNFvkBjKi6d_CkiXN7QdX36XfP-bSrrqKyh1l9NL8pF61RnyhTtlzafLXjp3DlT29NGqNJ281urq_xH7OREPS7OfSXuX0XsohR4kKFxd5K6C97XQAJ2jRkmELvtRq7WNoFEHR1RlFbPK7WE2JoSPSjz1LX0JIVrMYP4Q2wKl82Fetf6KoZ4DzuI6WDUH4eMhW6ijPoB9aCC2A9G_SAMI0qZA6KABUcPvhsJ2h-xAQ2jQeQPx8EwGPseYCqtNi_tk2xe5vEHZE3JvA)

I use cloudflare as my DNS provider, its free and secure and it has a lot of features. I use it to manage my domains. (Just the one lol :P) [devcoffee.me](https://devcoffee.me)

`Docker` and `docker-compose` have been instrumental in helping me manage all of my services in one place. Recently, I decided to switch to Kubernetes and installed k3s to make my life even easier.

I use `Gitea` as my self-hosted git repository because it gives me complete control over my code and allows me to manage my projects more effectively.

As a replacement for `Google Drive`, I switched to `Nextcloud` for hosting my files. This has been a great alternative for me since it offers a similar user experience while keeping my data private.

To download and manage torrent files, I use `Sonarr` and `Radarr`. These tools help me stay organized and keep track of all my downloads.

For monitoring all my services, I use `Uptime Kuma`. This tool helps me stay informed about the health of my infrastructure and quickly address any issues that arise.

On my Raspberry Pi, I have installed `Pi-Hole` to block ads and malware, making my browsing experience much safer.

As a media server, I use `Jellyfin`, which has been great for organizing my media collection and streaming it to various devices.

To browse files, I use `h5ai`, a simple file browser that allows me to easily view and access all of my files.

I use `Homer` as a dashboard to view all my services in one place. It has been a great tool for me to keep track of all my services and quickly access them.

`Ansible` helps me to automate the deployment of my infrastructure and scheduling tasks. Ex: regularly updating my os and installing security updates.

My CPU is connected to Printer, I use `CUPS` to manage my printer. I can print from any device connected to my network. (maybe using ngnix reverse proxy for this and expose to internet; is it worth it? lol)

I also plan to install `Jenkins` on my k3s cluster to automate my deployments or maybe use `argocd`, `Bitwarden` for password management, and a few other tools.

This was just a introduction to my personal infrastructure. Maybe I will be writing a series of blog posts about my personal infrastructure and how I built it.

I had a lot fun and learnt a lot while building my personal infrastructure, The best part is, there are always new tools and technologies to explore and integrate into my infrastructure. As I continue to evolve my personal infrastructure, I look forward to sharing my experiences and insights with others through my blog. Stay tuned for more!


Diving deep into my Personal Infrastructure


## Exchanging data between microservices

It's already 2023 and the software development is evolving lot faster than ever.

There are several software architecture patterns that are used to build a application, it depends on the use case and the requirements [majorly non functional] of the application.

The most popular architecture pattern is client-server architecture. It's a great way to build a scalable and maintainable application.
But the client-server architecture has its own challenges. One of the biggest challenges is running multiple services on a single server.
Most of the time, the server might not be powerful enough to run multiple services parellely, that's where microservices architecture comes into picture.

The microservices architecture is an subset of client-server architecture. It's a great way to build a scalable and maintainable application. But the microservices architecture has its own challenges. One of the biggest challenges is exchanging data between microservices. In this article, I will knot down some of the points which I learnt.

A common set of similar **methods** are grouped together to form a **microservice**. For example, [Ecommerce Application] a microservice can be a

- user service
- product service
- payment service
- shipping service, etc.

Each microservice can have its own database and it's own business logic. The microservices are independent of each other. They can be developed and deployed independently.
But the all the services need to communicate with each other at one point to be able to perform a task.
For example, when a user places an order, the user service needs to communicate with the product service to check the availability of the product and the payment service to process the payment.

user service --> product service --> payment service --> shipping service

This is where the challenge arises, **how can we exchange data between services internally?**

There are several ways to exchange data, it all boils down to the size, format and the frequency of data exchange and security requirements. [Better Applications can be developed from Good User requirements]

> Walking on water and developing software from a specification are easy if both are frozen.
> --Edward V. Berard

We'll take a look at 4 exchanging protocols [methods]:

- Synchronous Messaging<br/>
  REST APIs: A simple and common method to exchange data between microservices using HTTP requests and responses.

- Asynchronous Messaging: Using a message broker, such as Apache Kafka or RabbitMQ, to exchange messages between microservices asynchronously.

- gRPC: A high-performance, open-source framework for building remote procedure call (RPC) APIs.
- Event-Driven Architecture: Using events to trigger actions and exchange data between microservices.

Let's Try and take a look at each protocols Pro's and Con's

## 1. REST API's

This is a pretty common method to exchange data between frontend and backend services


Exchanging data between microservices


Not a lot of people might have heard of NTP,NTP is likely an essential part of many of the devices and services that you use on a daily basis. It helps to ensure that these systems have accurate and synchronized clocks, which is important for their proper functioning, Let's dive deep into this.

The Network Time Protocol (NTP) is a networking protocol that is used to synchronize the clocks of computer systems across a network.NTP is built on top of the User Datagram Protocol (UDP), which is a connectionless protocol that is used for transmitting data across a network.

This is important because many applications and services rely on accurate time to function properly.
For example, when you access a website, your computer needs to know the correct time to ensure that the secure connection is properly established.

NTP works by using a hierarchical structure of time servers to provide a reliable and accurate time source.

![image](https://www.researchgate.net/profile/Bogdan-Jeliskoski/publication/336902538/figure/fig2/AS:819773176832002@1572460564851/Detailed-hierarchy-of-the-Stratum-servers.ppm)

At the top of this hierarchy are the primary servers, which are connected to an accurate time source such as a GPS receiver or atomic clock. These primary servers then provide time information to secondary servers, which in turn provide time information to other computers on the network.

To use NTP, a computer will send a request to one of the time servers in the hierarchy. The server will then respond with the current time, along with information about the accuracy of its time source and any potential delays in the network. The computer will then use this information to adjust its own clock to match the correct time.

Main advantage of NTP is that it is a **decentralized protocol**, which means that it does not rely on a single time source. This makes it more resilient to network disruptions and outages, as the time information can be obtained from multiple servers.

Another advantage of NTP is that it can handle network delays and other types of interference. This is done through a process called **clock synchronization**, which uses algorithms to compensate for the effects of network delays and other factors that can affect the accuracy of the time signal.

### NTP Packet

When a computer wants to synchronize its clock with an NTP server, it sends a request packet to the server. This packet contains information such as the version of the NTP protocol being used, and the type of request being made.

When the server receives the request, it responds with a response packet. This packet contains the current time, along with information about the accuracy of the server's time source and any potential delays in the network.

The response packet also contains a number of fields that are used by the client to calculate the correct time. This includes the originate timestamp, which is the time at which the request was sent, and the receive timestamp, which is the time at which the request was received by the server.

Using these timestamps, along with the transmit timestamp, which is the time at which the response was sent, the client can calculate the round-trip delay, which is the amount of time it took for the request to be sent and the response to be received. The client can then use this information, along with the server's time, to adjust its own clock to match the correct time.

### Why UDP and not TCP

NTP uses UDP as its transport layer because it needs to send time information quickly and efficiently. NTP packets are typically very small, so the overhead of using a more complex protocol like TCP would be unnecessary.

Overall, the NTP packet is an essential part of the protocol, and is used to exchange time information between systems in order to synchronize their clocks. This helps to ensure that systems have accurate and synchronized time, which is important for many applications and services.

[Inspiration](https://www.youtube.com/watch?v=BAo5C2qbLq8&t=436s&ab_channel=Computerphile),
[Read More](https://blog.cloudflare.com/secure-time/),
[Read More](https://www.sciencedirect.com/topics/computer-science/network-time-protocol)


NTP- Network Time Protocol


Ever heard of AbortController, if not allow me to introduce you to the AbortController,

I came across AbortController in my company code base, at first I didn't understand why it was used or how it helps the application,
When I digged into it i was able to understand why it was used.

First of all let's see the MDN definition of Abort Controller:

The **`AbortController`** interface represents a controller object that allows you to abort one or more Web requests as and when desired.

In simple words,

It allows us to abort a network request that is currently in progress, and provides us with a few methods and properties to do so.

**AbortController** allows us to cancel a network request before it completes, which can help improve the performance of our application. This is especially useful in scenarios where we are making multiple network requests at once, and we want to cancel any that are no longer needed.

### Usage

To use the AbortController, we first need to create an instance of it. This is done using the `new` keyword, like so:

`const controller = new AbortController();`

Once we have an instance of the **AbortController**, we can use it's `signal` attribute to access the `abort` method. This method allows us to cancel the network request that is associated with the AbortController instance.

For Example if we are making `axios` request to the server we can simply pass the signal property of the **AbortController** like:

<script src="https://gist.github.com/Prajwalprakash3722/db45e29a0008cc9557ab0949a2a7ebee.js"></script>

In the code above, we are making a `axios` request to the `https://api.example.com/endpoint` URL, and passing the `signal` property of the `controller` instance as the `signal` option.

Now if we want to cancel the network request we can simple call the `abort` method on the `controller` instance.

`controller.abort()`

This will cause the request to be cancelled, and any pending promised chained up with the request will be rejected with **AbortError** so that we can catch the error and handle it appropriately.

### Diving deep into some additional Properties

The AbortController also has a `onabort` property, which is an event that is triggered whenever the `abort` method is called on the controller.

This allows to attach event listeners to the controller and perform certain actions when the network request is cancelled.

For example, if we want to display a message to the user whenever a network request is cancelled, we can attach an event listener to the `onabort` property of the controller, like so:

<script src="https://gist.github.com/Prajwalprakash3722/42d127fd5d10680b0703e123709b9f9f.js"></script>

### Practical Scenario:

Let's consider a application which fetched the number of friends and display them on the interface, let's assume **Rahul** has more number of friends and hence his network call will take more time, the user might not have patience to wait for the network call to be complete and presses **Rajus** name, **Raju** has few friends so his network call will be complete before **Rahul**'s network call.

After a while the network call made for **Rahul** friends resolves and the displaying UI will be replaced by **Rahuls** friends even the user purposefully clicked **Rajus** name.

Here **AbortController** will be helpful, when the user clicks the other name, we can just call the abort on the signal and cancel the request, hence increase the performance of the application as well.

### conclusion

The AbortController is a useful tool that allows us to cancel network requests in JavaScript. It provides us with the `abort` method, the `signal` property, and the `onabort` event, which give us **fine-grained control** over the network requests that are being made in the application.


new AbortController()


## Automating Your Postman Tests - 1

Postman is a collaboration platform for API development.
Postman's features simplify each step of building an API and streamline collaboration so you can create better APIs—faster.

This Post will be a Two part series, where we will be automating our Postman tests using the Postman API.
In this first part, we'll see about writing some **manual tests** in Postman, inspect responses, create workflows, create **collection** and a test suite
In the Second part we'll see how **CI/CD** pipeline can be setup for running the tests automatically.

This article requires you to have a prequestiste basic understanding of how HTTP/HTTPS works, Methods of HTTP, and how to perform simple GET and POST requests.

![Postman UI](public/assets/blog/automating-postman-tests-1/ui.png)

Postman needs a live HTTP server to process its requests. For this , we'll use one of my starter express backend setup with database, which is available [here](https://github.com/Prajwalprakash3722/express-ts-backend).

lets clone the starter application, install the necessary dependencies & start the server.

```bash

curl -4 localhost:5000

```

We should see a response like this

```json
{
  "ok": true,
  "message": "Hello from the Server"
}
```

Let's start using postman now :)

The backend starter template has a few routes setup, which we can use to test our postman tests.


Automating Your Postman Tests


## What I learnt from writing low level code for a semester

first of all let's try to understand what is `low level code`?,

From `Wikipedia`:

Low-level programming is a style of computer programming in which little or no abstraction is used between the instructions in the programming language and the architecture's machine code instructions.
Low-level programming is the opposite of high-level programming, which uses a large amount of abstraction to make the process of developing a program easier.

If thats a lot to consume let me simplify that for you:

- Low level code doesnt not use any abstraction; Meaning its the most basic form of code.
- Low level refers to code written directly for the computer.

I was one of those people who thought that low level language is the most difficult to learn and understand.
But after taking a course of `Micro Controllers and Embeded Systems` I can say that it is not that difficult to learn and understand.

Low level langauge requires you to think in a different way, it requires you to think in a way that you have to think about the hardware and the software at the same time.

It requires you to manage menory allocation and deallocation, you can also say thay in `c` you have to manage memory allocation and deallocation yourself.

But truth to be spoken `Assembly` & `Machine Code` are true low level languages.

High-level languages like `JavaScript` and `Python` may be the most popular programming languages today,
but there’s no argument that these languages are doing a lot of work for the developer,

Let's see some code comparisions between `ASM` and `c`:

## Fibbonaci Series in ASM

<script src="https://gist.github.com/Prajwalprakash3722/15a14dc3aa379b276a29f0d3f98e59ab.js"></script>

## Fibbonaci Series in C

<script src="https://gist.github.com/Prajwalprakash3722/9f49eb73185b66900a26b5cc4a983e53.js"></script>

of course the no of lines of code is more in `asm`, code is not that readable and it is not that easy to understand.

why? because we are writing code directly for the computer, we are not using any abstraction, libraries.
no of keywords reserved in `asm` is less than `c` and `c++`. This again makes it difficult to understand.

I found it extremely unproductive and it requires you to do a lot of work for very little results.
It does however have a good side. It is very fast and it is very efficient, memory safe.

I also used c to write Embeded code for a semester, I used `LPC1248` microcontroller and `C` to write code for it.
here `arm` was also very much useful as we knew which memory location was used for which purpose, we knew which registers were used for which purpose.

Couple things I observed was that `c` is very much similar to `ARM` in terms of syntax, it is just that `c` has a lot of abstraction and `c` is a lot more readable than `ARM` and the `ARM` code ran insanely fast compared to `c` code.

## My Final Thoughts

I think that low level languages are not that difficult to learn and understand, it just requires you to think in a different way.
concept of Registers, memmory addresses and memory allocation and deallocation is very important to understand.
You gain a pratical expirence of how the computer works.

I think every software developer has to learn if not how to write but atleast to read and understand low level code.
It explains how a processor works from the inside.


What I learnt from writing low level code for a semester


## Heroku Alternatives

Heroku was one one of the earliest cloud platforms to offer a PaaS (Platform as a Service) model.

It was founded in 2007 and has been a leader in the PaaS space ever since.
Heroku is a great platform for developers to deploy their applications and scale them as needed.

However, Heroku is not the only PaaS platform out there. In this article, we will look at some of the best Heroku alternatives
as it has announced to stop supporting the free tier (hobby) plan from November 28th 2022.You can read more about this [here](https://blog.heroku.com/next-chapter/).

I still remember the first time I deployed an application on Heroku, It was so easy to deploy.

It was a simple Flask application that I had cloned from [hastebin](https://hastebin.com/), I still think its deployed somewhere on Heroku. [hastebin-clone](https://haste-bin-clone.herokuapp.com/)

It literally deployed any code I threw at it that too for free. I was amazed at how easy it was to deploy an application on Heroku.It was essiential to any developer starting out.

Earlier in the days it was not easy to run code on server, it required a special set of knowledge and skills to run code on a server.

for ex;

- Have a VM hosted somewhere with a public Ip address alloted to it.
- SSH into it
- Install the necessary software/dependeices
- Ingress & Egress rules
- Firewall rules
- Reverse proxy if the server was mapped to any domain
  and the list goes on.

## What was hobby tier?

Hobby tier was a free tier that Heroku offered to developers to deploy their applications for free.
It offered 512 MB of RAM and 1 dyno. It was a great way to get started with Heroku.
we could attach any VCS (Version Control System) to our application and deploy it with a single click.
we could also get a smol database for free.

Well from that day most of my personal projects have been deployed on Heroku on hobby tier.

Now considering that they'll be stopping the free tier from November 28th 2022, I thought it would be a good idea to look at some of the best Free Heroku alternatives.

as Heroku killed the free tier they literally digged out a hole for themselves; I doubt Heroku will be able to compete with the other PaaS platforms out there.

## host a website (Frontend)

If you are looking to host a website for free free,

- Github Pages
- Netlify
- Vercel
- Render
- Surge
- Cloudflare Pages
- Firebase hosting

## Host a website (Backend)

If you are looking to host a backend for free,

- Railway.app
- Repl.it
- Render
- Cloudflare Workers

## Host a docker Image

Nowdays most of the applications are containerized and deployed as docker images across the network to facilitate for easy deployment and scaling.

- Railway.app
- Render
- Fly.in
- Digital Ocean App Platform

## Host a database

- Railway.app
- PlanetScale
- Supabase
- MongoDB Atlas
- Firebase Realtime Database
- Firebase Firestore

Any way there are endless opportunities out there, I tried to complie the most frequent services I use and the ones which offer easy deployment and scaling, you can host your application on any of the above platforms for free.

Ofc there is the 2 gaints in the market, AWS and GCP, they offer a lot of services (all of the above) for free, but they are not as easy to use as the above platforms. '
That's why I didn't include them in the list.

## Conclusion

There are **so many good** alternatives out there to deploy a full stack application.
If you are looking to deploy a full stack application for free, I would recommend you to try out [Railway.app](https://railway.app/).
It's one of the best Heroku alternatives out there.

If I missed out on any good Heroku alternatives, please let me know, [Mail Here](mailto:prajwalprakash3722@gmail.com)


Heroku Alternatives


## learning_rust[0]

After a heated up discussion with myself & my brain about which static typed langauge to learn,
I was thinking about learning `Go` & `Rust` for a long time but I was not sure which one to learn first,
I finally decided to learn Rust first because I think it is more suitable for my current skill level.

What do I mean by that? I mean that I am not a beginner in programming but I am not an expert either,

But I have a fair understanding of programming concepts & I have a good understanding of functional programming.
I like tinkering with operating systems, processes, threads, etc..
Curious about how memory management work under the hood.

![Ferris](https://miro.medium.com/max/450/1*LBHHLYrwLCSD0CYu-rfWMA.png)

So I thought `Rust` is a good fit for me, I am not sure if I am right or not but I will find out soon.

Let's start with the basics,
I am going to learn Rust by following the official manual [The Rust Programming Language](https://doc.rust-lang.org/book/title-page.html).

Imma try to write a post for 3 chapter each time, so I can keep track of my progress and can share my experience.

PS: This is what motivated more to learn Rust xD; <br/>
`In most situations, we’ll lead you to the correct version of any code that doesn’t compile.`

![The Rust Programming Language](https://i.ibb.co/yVSS22N/Screenshot-from-2022-09-16-22-33-54.png)

## Some Intresting things about Rust I found when I was pouring through the web

- Rust’s unofficial Mascot is an orange crab named Ferris
- Karen Rustad Tölva drew the mascot, and Rust is part of her name.(Credit: Reddit)
- Rustacean is a term for Rust programmers.


learning_rust[0]


## Don't get confused between map() and forEach()

JavaScript is the one of the easiest langauge to learn and as well as the most confusing langauge ever.

I have been using JavaScript for more than 2 years and I have missed out of a fair share of features, misunderstood a lot of things.

One of the most common mistakes I have seen is people (including me) using map() and forEach() wrong.

Now JavaScript offers us two methods to iterate over an array, `map()` and `forEach()`.

## Let's see what MDN documentation says about them.

_The `Array.prototype.forEach()` method executes a provided function once for each array element._

What this means ?

The function that is passed as a parameter to the forEach() method will be executed for each element in the array.

_The `Array.prototype.map()` method creates a new array populated with the results of calling a provided function on every element in the calling array._

What this means ?

The function that is passed as a parameter to the map() method will be executed for each element in the array and the return value of the function will be added to the new array which will be returned by the map().

## whoa thats a lot to consume, let's break it down.

## forEach()

forEach() is a method that is used to iterate over an array and execute a function for each element in the array.

Let's see an example.

<script src="https://gist.github.com/Prajwalprakash3722/bccc97b75210f74a6af42090d84bf1e3.js"></script>

In the above code, we can see that a anoymous function (arrow functions) is passed as a parameter to the forEach() method.
However, instead of returning a new array like map, it returns `undefined`.

## map()

map() is a method that is used to iterate over an array and execute a function for each element in the array and return a `new array`.

Let's see one more example.

<script src="https://gist.github.com/Prajwalprakash3722/4d0ab52d25743ef1a7e4200530130b98.js"></script>

In the above code, we can see that a anoymous function is passed as a parameter to the map() method.
This method will return a new array which we are using to print the values back to the console.

## Any other differences ?

If you have heard the concept of chaining in JavaScript, then you might have heard of the term `method chaining`.

bascially in a nutshell `method chaining` is a way to call multiple methods on the same object.

You cannot attach multiple methods with forEach() as it returns undefined.

But you can chain many methods such as filter(), map(), reduce() etc with map() as it returns array.

Smol example code to see method chaining.

<script src="https://gist.github.com/Prajwalprakash3722/c180fc95434ecbda33ec18059587d0ed.js"></script>

## Which of em is speed ???

Both of em are equally speed, but I have seen people saying map() is more performant than forEach().

But ultimatetly it depends on the situation,

If you plan to use the returned array then use map() or else; just use forEach().

## Smol Table for better Conclusion

<table className="">
  <thead className="">
    <tr>
      <th>forEach()</th>
      <th>map()</th>
    </tr>
  </thead>
  <tbody id="tbody">
    <tr>
      <td>Iterates over an array</td>
      <td>Iterates over an array</td>
    </tr>
    <tr>
      <td>Does not return anything</td>
      <td>Returns a new array</td>
    </tr>
    <tr>
      <td>Cannot chain methods</td>
      <td>Can chain methods</td>
    </tr>
  </tbody>
</table>


Don't get confused between map() and forEach()

Generics in TypeScript


# How Email Works ?

Recently I was building a Hobby Project which was entirely based on emailing services, I decided to host my own Email server to avoid extra costs and But a little I did know was Emailing is a RatHole.

When I dived into it more, I realised that my understanding of the **entire emailing conccept** wrong; over the time I tried to understand what email is excatly and how it works.

let me share with y'all :)

Ever wondered How Emails Work, Who are delivering the electronic letters or more known as "Emails"

When You send a email there is a lot that happens in the background to send to recipitant,

Email is a system for sending and receiving messages over the internet. It works by using a protocol called Simple Mail Transfer Protocol (SMTP) to send messages between servers.

When a user wants to send an email, they can use a client program, such as Microsoft Outlook or Apple Mail, to compose and send the message. The client program connects to the user's email server, which is a computer that stores the user's email messages and handles sending and receiving messages.

The email server uses SMTP to send the message to the recipient's email server. The recipient's email server then delivers the message to the recipient's email client, which is the program they use to access their email.

In addition to SMTP, other protocols are also involved in the email process. For example, the Internet Message Access Protocol (IMAP) and the Post Office Protocol (POP) are used to retrieve messages from the email server and download them to the user's email client.

To simplify, we can say that the email client is the program that users use to access and manage their email, and the email server is the computer that stores and handles the sending and receiving of messages. The protocols, such as SMTP, IMAP, and POP, are the technical rules and instructions that enable the email client and server to communicate and exchange messages.



## Namespaces in Typescript

In this article we will try to grasp the concept of namespaces and how they are helpful and
finally we will learn how to use them in Typescript.

[directly jump to coding->](#code)

PS: Typescript is my go to language all time [here](https://blog.devcoffee.me/posts/typescript-javascript) is why.

First of all what are namespaces ?

As per wikipedia definition

_In computing, a namespace is a set of signs (names) that are used to identify and refer to objects of various kinds. A namespace ensures that all of a given set of objects have unique names so that they can be easily identified._

Namespaces are used to group related

- functions
- interfaces
- classes

Namespaces are used to avoid naming conflicts, organize code into logical groups and to provide privacy for variables and functions.

Concept of namespace is not new, we have been using them in some databases such as cassandra,

Linux Kernel also uses namespaces to isolate the processes from each other;
thus allowing high level fine-grain partitioning of resources.

It uses namespaces to isolate the network, mount, user, pid, uts, ipc, cgroup etc.. (Maybe we can cover this in another episode)

Now you might ask Why Namespaces at all ?

Lets imagine namespaces as a _Box_, where we can put all the related stuff in it.
Now we can use this box to share it with other people.
But we can also keep it private and use it for our own purpose.
You can add some crazy stuffs to the box but the thing is it will not affect content of other boxes.

This helps making the code more orginised & clean.

Smol excerpt on adavantages of namespaces:

- Code Reusability
- Distributed Development

### <a id="code">How to use namespaces in Typescript ?</a>

```typescript
namespace RandomNameSpaceName {}
```

Smol Code Example to show how we can use same variable name, same funcion name in different namespaces.

<script src="https://gist.github.com/Prajwalprakash3722/9a211072354cf6548776e54be38bc233.js"></script>

The above code allows us to use the same variable and function name without collision.

## But How to access the whatever we have defined in the namespaces ?

To access the functions, variables, classes, interfaces, etc. we have to use the `export` keyword before the function or classname.

<script src="https://gist.github.com/Prajwalprakash3722/94f56b8a7b54cd5eb21779ada675d644.js"></script>

Let's see a simple & pratical real world application implementation of namespaces in Typescript.

Assume that we have a To-Do application.

<script src="https://gist.github.com/Prajwalprakash3722/a96c578ffdb1932d93d6c1f35263247a.js"></script>

In the above code we have declared all the possible interfaces that we might use in the application.

Note that We have 3 levels of namespaces

- TodoApi
- Controllers
- Add/Get

Now let's use the decalared namespaces in the application. (ofc application doesn't work; the code is just for demonstration)

<script src="https://gist.github.com/Prajwalprakash3722/016b07c3a756cb6d382934e9ac99521e.js"></script>

You can understand the code whole lot better, its cleaner, more orginised and easy to maintain.

## At last namespaces are just set of names that are used to identify and refer to objects of various kinds.


Namespaces in Typescript


Idempotent you must have heard of this word somewhere in your software journey, what is it? how does it affect us, how one can build Idempotent Systems?

But instead of giving the definition of `Idempotent` I want you to imagine a situation.

Now, let's imagine a situation where you're broke and hungry, and you decide to order food from Swiggy. You have a bank balance of only 400/-, and you select paneer butter masala and roti,
which costs 350/-. After filling in all the necessary information, you go ahead with the order. However, for some mysterious reason, the order fails.

There could be a few reasons why the order failed:

1. The order request didn't even reach the server due to network interruptions. In this case, the client might be aware of the error and can attempt to retry the order.

2. There was an internal server error (5xx) that prevented the order from being processed. Unfortunately, the client has no way of knowing this error and might mistakenly assume that the order failed.

3. The order was successfully processed, but the response packet got lost in a network interruption. Again, the client has no way of knowing this and may attempt to place the order again.

In this situation, if the error was due to the first reason, refreshing the page can save you from spending another 350/-. However, if it was due to the second or third reason, refreshing the page would lead to duplicate orders and potentially additional charges.

Modern systems often employ automatic retries to improve user experience and increase the chances of successful communication. By designing systems with idempotence in mind, we can remove the impact of failures and provide a more reliable and consistent UX for users.

Systems built with idempotence in mind can recover from failures, retry requests, and maintain data integrity without any inconsistencies.

Now the formal definition Idempotent is a term used in software development to describe an operation or function that can be applied multiple times without changing the result beyond the initial application. In other words, performing the operation multiple times should have the same effect as performing it once. (I didn't write the definition lol)

In the domain of HTTP methods, which are widely used in web development, we commonly encounter four main methods: `GET`, `POST`, `PUT`, and `DELETE`. These methods can be categorized into idempotent and non-idempotent based on their behavior.
Idempotent methods, such as `GET` and `PUT`, have the property that making multiple identical requests will not lead to different outcomes or modify the server's state. On the other hand, non-idempotent methods, like `POST`, are designed to create new resources making them non-idempotent by nature.

Now let's see a small example of how to build a idempotent API in Spring Boot :)


Building Resillent & Idempotent System using Spring Boot


Ever wondered how does JavaScript sorts the array when you call `Array.sort()` ?

Well It all depends on the runtime engine of the browser.

for example,

- Chrome uses V8 engine, which is a JavaScript engine written in c++. As per their documentation
  [V8](https://v8.dev) is Google’s open source high-performance JavaScript and WebAssembly engine, written in C++. It is used in Chrome and in Node.js, among others. It implements ECMAScript and WebAssembly, and runs on Windows 7 or later, macOS 10.12+, and Linux systems that use x64, IA-32, ARM, or MIPS processors. V8 can run standalone, or can be embedded into any C++ application.

- Firefox uses SpiderMonkey engine, which is a JavaScript engine written in C++. As per their documentation
  [SpiderMonkey](https://spidermonkey.dev) is Mozilla’s JavaScript and WebAssembly Engine, used in Firefox, Servo and various other projects. It is written in C++, Rust and JavaScript. You can embed it into C++ and Rust projects, and it can be run as a stand-alone shell.

- Safari uses JSC engine, which is a JavaScript engine written in C++. As per their documentation
  [JSC](https://webkit.org/) WebKit is the web browser engine used by Safari, Mail, App Store, and many other apps on macOS, iOS, and Linux.

- Microsoft Edge uses Chakra engine, which is a JavaScript engine written in C. As per their documentation
  [ChakraCore](https://github.com/chakra-core/ChakraCore) is a JavaScript engine with a C API you can use to add support for JavaScript to any C or C compatible project. It can be compiled for x64 processors on Linux macOS and Windows. And x86 and ARM for Windows only. It is a future goal to support x86 and ARM processors on Linux and ARM on macOS.

We will take the 2 most popular browsers as reference, and compare the performance of `Array.sort()` in each of them.

- ## Google Chrome

  Since v8 engine Chrome has been using TimSort algorithm for sorting the array.

  ![avatar](https://www.openbookproject.net/books/bpp4awd/_static/ch10/timpeters.jpg)

  TimSort developed by <u>Tim Peters</u> in 2002 for [python](https://github.com/python/cpython/blob/main/Objects/listsort.txt).
  While Mergesort usually works in recursive fashion, Timsort works iteratively.

  The Array is divided into Run blocks.
  These runs are sorted using insertion sort one at a time, and they are then combined using the combine function from merge sort.
  The Array is sorted solely using Insertion Sort if its size is smaller than run block.
  Depending on the size of the array, the run's size can range from 32 to 64.

## Algorithm:

1. Divide the array into the number of blocks known as run.
2. Consider the size of run, either 32 or 64 (32 is the default).<br/>
3. Sort the individual elements of every run one by one using insertion sort.
4. Merge the sorted runs one by one using the merge function of merge sort.
5. Double the size of merged sub-arrays after every iteration.

WKT the basic operation of insertion sort is Comparing.

When we merge the sorted lists, we come up with a total n-1
comparison because the last element which is left will need to be copied down in the combined list,
and there will be no comparison.


Analyzing Array.sort() & measuring the performance in JavaScript


So a Little intro to Typescript, It was developed by Microsoft as a opensource project and its now also maintained by Microsoft.

So Basically in a nutshell Typescript is the super set of JavaScript meaning any valid JavaScript code is a valid Type-Script Code.

If you have experience with JS you know that the data type is completely dynamic, to give you a example, I declare `arr` as a const , and try to apply `arr.map()` function, This is a error because `arr` is not a array to map through and we’ll get to know this error during run (JS doesn’t need any compiling we can directly run / interpret it in Browser) time which is more frustrating for a developer

Now, lets come to typescript you have to mention the datatype of `arr` which is a string and if you try to apply `arr.map()` the text editor will scream at you saying that you cannot loop though a string….

TypeScript is strictly better than JavaScript. Some of the advantages are below,

- Static types prevent an entire category of errors. And they prevent the errors before you even save the code to disk. The sooner you detect an error, the cheaper it is to fix in developers point of view.

- It forces you to write clean code, so if you forget your code after a couple months you can just see what you have written and what does this do in just couple minutes..

- If you are building a large Systems, Its Oriented and complex systems is no longer a nightmare.

## Compilation

A main dIffrence between JavaScript and TypeScript is compilation. There is no need to build JS code. In the browser, it is interpreted and executed. TypeScript code, on the other hand, must be converted into standard JS and then compiled. It may seems that you might be wasting time, but you are actually saving time. Compiling the code takes a few extra seconds, but it saves time when you are testing your application.

After everything is said and done, both Typescript and JavaScript have their own set of limitations and features. JS is a dynamic and lightweight coding language designed to improve HTML web pages. It is not, however, a full-fledged scripting language. JS is an interpreted programming language that runs in the context of a web browser.

Typescript is compiled to JS, which can be used in any JS code; as a result, Typescript has more advantages, making it more significant and popular.

## Final Notes:

The community for Typescript is not big as JS

Use TS always whenever you are making a big application with complicated Data and interacting with various Databases

You can get more accurate Auto completions in your text editor as the transpiler knows what data type it is

Start Migrating your codebases to TS, its easy just renaming the file extensions to .ts` Of course, JavaScript code won’t become a strict TypeScript automatically. It still takes some time and effort to convert.But its so much worth it to make it.

Just use Ts, You’ll thank me later xD.


← back

Software Development related Posts

2023

2022